Technology Assisted Reviews: Finding the Last Few Relevant Documents by Asking Yes/No Questions to Reviewers

The goal of a technology-assisted review is to achieve high recall with low human effort. Continuous active learning algorithms have demonstrated good performance in locating the majority of relevant documents in a collection, however their performance is reaching a plateau when 80\%-90\% of them has been found. Finding the last few relevant documents typically requires exhaustively reviewing the collection. In this paper, we propose a novel method to identify these last few, but significant, documents efficiently. Our method makes the hypothesis that entities carry vital information in documents, and that reviewers can answer questions about the presence or absence of an entity in the missing relevance documents. Based on this we devise a sequential Bayesian search method that selects the optimal sequence of questions to ask. The experimental results show that our proposed method can greatly improve performance requiring less reviewing effort.Comment: This paper is accepted by SIGIR 201

The best of the UK? A report on the value and future of UK databases in the health and social care fields: a systematic map protocol

INTRODUCTION: This protocol covers the first part of a two-part project funded by the Health Libraries Group and the University Health and Medical Librarians Group. It details the proposed methodology for a systematic map of the literature relating to UK bibliographic databases in the fields of health and social care. The aim of this mapping exercise is to consider ways in which UK bibliographic databases are described, considered and discussed in the published and unpublished literature. In doing so, we hope to gain a clearer sense of the ways in which UK bibliographic databases are used and viewed by the research community. It also enables the identification of any gaps in the literature for further research and discussion. This topic is important because UK databases are generally underused by researchers in the UK context and some databases are at risk of closure. A lack of access to UK databases means that researchers may miss relevant UK evidence when identifying an evidence base. / METHOD: Systematic Map. / ANALYSIS: The authors will present a narrative description of the literature relating to UK bibliographic databases in the fields of health and social care. They will use tables to present descriptive information about the literature (eg, frequency tables) and use cross-tabulations to demonstrate intersecting themes. Separately, guidance on how to use the resources (eg, areas of unique content, updating frequencies, unique truncation symbols) will be sought from stakeholders and reported alongside the report narrative as a guide to usage

The Value of Co-Production Research Project: A Rapid Critical Review of the Evidence

What are the different types of co-production within research and what are their value(s)

Plain Tobacco Packaging: A Systematic Review

(From the Executive Summary): This systematic review outlines findings from 37 studies that provide evidence of the impacts of plain tobacco packaging. The review was conducted following the publication of the March 2011 White Paper Healthy Lives: Healthy People which set out a renewed Tobacco Control Plan for England. One of the key actions identified in the plan was to consult on possible options to reduce the promotional impact of tobacco packaging, including plain packaging. This systematic review was commissioned to provide a comprehensive overview of evidence on the impact of plain packaging in order to inform a public consultation on the issue

A review of the effectiveness of smokefree strategies and interventions in secondary care settings

The aim of this review was to systematically review the effectiveness of smokefree strategies and interventions in secondary care settings (acute, maternity and mental health settings). The initial search and screening stages were combined with a parallel review of the barriers to and facilitators for implementing smokefree strategies and interventions in secondary care settings conducted by members of the same research team.The review aimed to address the following questions:Question 1: How effective are strategies and interventions for ensuring compliance with smokefree legislation and local smokefree policies in secondary care settings?Subsidiary question: How does the effectiveness vary for different population groups, health status or speciality care services?Question 2: Are there any unintended consequences from adopting smokefree approaches in acute and maternity care settings?Question 3: Are there any unintended consequences from adopting smokefree approaches in mental healthcare settings?As the extent of evidence on the effectiveness of smokefree strategies was limited to two studies for Question 1, the data are also presented from identified effectiveness studies with a comparative design to measure indicators of compliance in settings which had a smokefree policy with at least one supporting strategy covering the whole estate or an indoors-only policy

A review of the barriers to and facilitators for implementing smokefree strategies and interventions in secondary care settings

The aim of this review was to systematically review the barriers to and facilitators for implementing smokefree strategies and interventions in secondary care settings (acute, maternity and mental health settings) from service users' and service providers' perspectives. The initial search and screening stages were combined with a parallel review of the effectiveness of smokefree strategies and interventions in secondary care settings conducted by members of the same research team.This review aimed to address one overarching question; what are the barriers and facilitators affecting adoption of, support for, and compliance with smokefree policies in secondary care settings?; and was guided by three subsidiary questions: How does support for smokefree policy differ by population group, service provider and type of policy? What factors have an impact on acceptance of smokefree policies? What are the adverse events and other consequences associated with smokefree policies

Using systems perspectives in evidence synthesis: A methodological mapping review

BACKGROUND: Reviewing complex interventions is challenging because they include many elements that can interact dynamically in a non-linear manner. A systems perspective offers a way of thinking to help understand complex issues, but its application in evidence synthesis is not established. The aim of this project was to understand how and why systems perspectives have been applied in evidence synthesis. METHODS: A methodological mapping review was conducted to identify papers using a systems perspective in evidence synthesis. A search was conducted in seven bibliographic databases and three search engines. RESULTS: A total of 101 papers (representing 98 reviews) met the eligibility criteria. Two categories of reviews were identified: 1) reviews using a 'systems lens' to frame the topic, generate hypotheses, select studies, and guide the analysis and interpretation of findings (n=76) and 2) reviews using systems methods to develop a systems model (n=22). Several methods (e.g., systems dynamic modeling, soft systems approach) were identified and they were used to identify, rank, and select elements, analyze interactions, develop models, and forecast needs. The main reasons for using a systems perspective were to address complexity, view the problem as a whole, and understand the interrelationships between the elements. Several challenges for capturing the true nature and complexity of a problem were raised when performing these methods. CONCLUSION: This review is a useful starting point when designing evidence synthesis of complex interventions. It identifies different opportunities for applying a systems perspective in evidence synthesis, and highlights both commonplace and less familiar methods

Effective teacher professional development: new theory and a meta-analytic test

Multiple meta-analyses have now documented small positive effects of teacher professional development (PD) on pupil test scores. However, the field lacks any validated explanatory account of what differentiates more from less effective in-service training. As a result, researchers have little in the way of advice for those tasked with designing or commissioning better PD. We set out to remedy this by developing a new theory of effective PD based on combinations of causally active components targeted at developing teachers’ insights, goals, techniques, and practice. We test two important implications of the theory using a systematic review and meta-analysis of 104 randomised controlled trials, finding qualified support for our framework. While further research is required to test and refine the theory, we argue that it presents an important step forward in being able to offer actionable advice to those responsible for improving teacher PD

Recalibration as an approach to context in statistical meta-analyses

Background: Using evidence from meta-analyses for local decision-making is hampered by the lack of explicit connection between the contexts in which interventions were evaluated and the context in which the evidence is to be applied. New methods are needed to address the question, ‘is there any evidence that the intervention will work differently in an area like mine?’ / Methods: We reanalyzed a meta-analysis of school-based interventions to reduce fat intake. Using observational data, effect sizes were reweighted to reflect the contextual similarity of studies to four UK Local Authorities. Studies that were more like Local Authorities contributed more towards the pooled effect size. / Results: The original meta-analysis found that the intervention was not effective in reducing fat intake. In the recalibration analyses, because contextually relevant studies showed greater effects, the recalibrated pooled effect sizes indicated a larger effect with a narrower confidence interval in three Local Authorities, but the interpretation remained unchanged in a fourth. The findings held under both fixed effect and random effects specifications. / Discussion: For three of the local authorities examined, we have greater confidence that school-based interventions will have an impact on reducing fat intake. Recalibration may give a contextually-nuanced estimate for a given local context

Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews.

In scoping reviews, boundaries of relevant evidence may be initially fuzzy, with refined conceptual understanding of interventions and their proposed mechanisms of action an intended output of the scoping process rather than its starting point. Electronic searches are therefore sensitive, often retrieving very large record sets that are impractical to screen in their entirety. This paper describes methods for applying and evaluating the use of text mining (TM) technologies to reduce impractical screening workload in reviews, using examples of two extremely large-scale scoping reviews of public health evidence (choice architecture (CA) and economic environment (EE)). Electronic searches retrieved >800,000 (CA) and >1 million (EE) records. TM technologies were used to prioritise records for manual screening. TM performance was measured prospectively. TM reduced manual screening workload by 90% (CA) and 88% (EE) compared with conventional screening (absolute reductions of ≈430 000 (CA) and ≈378 000 (EE) records). This study expands an emerging corpus of empirical evidence for the use of TM to expedite study selection in reviews. By reducing screening workload to manageable levels, TM made it possible to assemble and configure large, complex evidence bases that crossed research discipline boundaries. These methods are transferable to other scoping and systematic reviews incorporating conceptual development or explanatory dimensions